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Continuous- Time Equality- Constrained Optimization Based on a New Projection 

Matrix 

Quan Quan and Kai-Yuan Cai 
Abstract 

We propose a new projection matrix, based on which a continuous-time, equality-constrained 
optimization approach is developed. First, the equality constraint is transformed into a continuous- 
time dynamical system with solutions that always satisfy the equality constraint. A new projection 
matrix is then proposed to avoid singularity in the dynamical system. A controller is subsequently 
designed to decrease the objective function along the solutions of the transformed system. The 
invariance principle is applied to analyze the behavior of the solution. We also propose a modified 
approach for addressing cases in which solutions do not satisfy the equality constraint. Finally, the 
proposed optimization approaches are applied to two examples to demonstrate its effectiveness. 

Index Terms 

Optimization; equality constraints; control theoretic approaches; continuous-time dynamical sys- 
tems 

I. Introduction 

According to the implementation of a differential equation, most continuous-time optimization 
approaches can be classified as either a dynamical system fj_,f__,__] or a neural network [41.l51.f6ll.f7l. 
The dynamical system approach relies on the numerical integration of differential equations on a 
digital computer. Unlike discrete optimal methods, the step sizes of dynamical system approaches 
can be controlled automatically in the integration process and can sometimes be made larger than 
usual. This advantage suggests that the dynamical system approach can in fact be comparable with 
currently available conventional discrete optimal methods and facilitate faster convergence ifll .OI. The 
application of a higher-order numerical integration process also enables us to avoid the zigzagging 
phenomenon, which is often encountered in typical linear extrapolation methods On the other 
hand, the neural network approach emphasizes implementation by electrical circuit, very large scale 
integration, and optical technologies (_]. The major breakthrough of this approach is attributed to 
the seminal work of Hopfield, who introduced an artificial neural network to solve the traveling 
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salesman problem (TSP) [9). By employing analog hardware, the neural network approach offers low 
computational complexity and is suitable for parallel implementation. 

For continuous-time equality-constrained optimization, existing methods can be classified into three 
categories ID: feasible point method (or primal method), augmented function method (or penalty 
function method), and Lagrangian multiplier method. Determining whether one method outperforms 
the others is difficult because each method possesses distinct advantages and disadvantages. Readers 
can refer to |[Tl.ll4l.ir71l. |[T0l and the references therein for details. The feasible point method directly 
solves the original problem by searching through the feasible region for the optimal solution. Each 
point in the process is feasible, and the value of the objective function constantly decreases. Compared 
with the two other methods, the feasible point method offers three significant advantages that highlight 
its usefulness as a general procedure that is applicable to almost all nonlinear programming problems 
iTTOl p. 360]: i) the terminating point is feasible if the process is terminated before the solution is 
reached; ii) the limit point of the convergent sequence of solutions must be at least a local constrained 
minimum; and iii) the approach is applicable to general nonlinear programming problems because it 
does not rely on special problem structures such as convexity. 

In this paper, a continuous-time feasible point method is proposed for equality-constrained optimiza- 
tion on the basis of a feedback control perspective. Local convergence results do not assume convexity 
in the optimization problem to be solved. Compared with the discrete optimal methods offered by 
Matlab, at least two illustrative examples show that the proposed method can avoid convergence 
to a singular point and can facilitate faster convergence through numerical integration on a digital 
computer. The major contributions of this paper are as follows: i) a new projection matrix is proposed 
to remove a standard regularity assumption that is often associated with feasible point methods, 
namely, that the gradients of constrains are linearly independent, see H] p.158, Equ.(4)],[2, p. 156, 
Equ. (2.3)1,17, p. 1669, Assumption 1]; ii) based on the proposed matrix, a continuous-time, equality- 
constrained optimization is developed to avoid convergence to a singular point, and the modified 
version of the proposed optimization is further developed to address cases in which solutions do 
not satisfy the equality constraint; and iii) equality constraints are transformed into continuous-time 
dynamic systems, which can serve as a controlled plant for control-theoretic approaches ifTTl to study 
equality-constrained optimization. 

We use the following notation. W 1 is Euclidean space of dimension n. ||-|| denotes the Euclidean 
vector norm or induced matrix norm. I n is the identity matrix with dimension n. niXri2 denotes a 
zero vector or a zero matrix with dimension ri\ x n^. Direct product <g) and vec(-) operation are 
defined in Appendix A. The function [-] x : R 3 — > M 3x3 with matrix H G M 9x3 is defined in Appendix 
B. Suppose g : R n — > R. The gradient of the function g is given by Vg (x) = V x g (x) = [dg (x) /dx\ 
■ ■ ■ dg (x) /dx n ] T € W 1 and the matrix of second partial derivatives of g (x) is known as Hessian 
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given by V ' xx : R -> R nxn and V^p (x) = [<9 2 # (x) /dxidxj).. . 

II. Problem Formulation 

A. Equality-Constrained Optimization 

A class of equality-constrained optimization considered here is depicted as follows: 

minw(x), s.t. c(x)=0 (1) 

where v : R n — >• R is the objective function and c = [ci eg * * * c m ] T G R m , Cj : R n — > R are the 
equality constraints. They are both twice continuously differentiable. To avoid a trivial case, suppose 
the constraint (or feasible set) F = {x G R n | c (x) = 0} ^ 0. 

Definition 1 lfT2l pp. 316-317]. For the problem £0), a vector x* € J 7 is a global minimum if 
v (x*) < v (x) , Vx E J 7 ; a vector x* G J 7 " is a local (strict local) minimum if there is a neighborhood 
M of x* such that v (x*) < v (x) (v (x*) < v (x)) for x G A/" n 7". 

Definition 2 iflOl p. 325]. A vector x* G J 7 is said to be a regular point if the gradient vectors 
Vci (x*) , Vc2 (x*) , • • • , Vc m (x*) are linearly independent. Otherwise, it is called a singular point 
here. 

This paper aims to propose a continuous-time, equality-constrained optimization approach to iden- 
tify the local minima based on a feedback control perspective. 

B. Equality Constraint Transformation 

Optimization problems are often solved by using numerical iterative methods. For an equality- 
constrained optimization problem, the major difficulty lies in ensuring that each iterative satisfies 
the constraint and can move toward the minimum. To address this difficulty, a transformation of the 
equality constraint is proposed, which is formulated as an assumption. 

Assumption 1. For a given xo G J 7 , there exists a function / : M n — > M. nxl such that 

x(t) = f(x(t))u(t),x(0) = x o (2) 

with solutions that satisfy x (t) G T u (xq) , where T u {xq) = {x (t) G F\x (t) = f(x(t))u(t), 
x (0) = x G T, Vu (t) G M 1 , t > 0}. 

From a feedback control perspective, u can be considered as a control input. The objective function 
v (x) can be considered a Lyapunov-like function, although v(x) is not required to be a Lyapunov 
function. Based on Assumption 1, the objective of this paper can be restated as: to design a control 
input u to decrease v(x) along the solutions of ([2]) until x has achieved a local minimum. In the 
following, we will omit the variable t except when necessary. 

Remark 1. The problem formulation here is more general than that proposed in ifTTIl . First, the 
considered problem is an optimization problem rather than a zero finding problem. Secondly, the 



September 25, 2012 



DRAFT 



4 



continuous-time dynamical system © can degenerate to x = u by setting I = n and / (x) = I n , 
namely an unconstrained optimization problem considered in [11]. For such a case, we have T u {%o) = 
R n because the system x = u is controllable. 

Remark 2. The proposition of Assumption 1 is motivated by the property of attitude kinematics 
1131 p. 200]: x = \E{x)w, where x = [q q T ] T G M 4 , q G R, q,w £ M 3 and # (x) = [-g T 
<?o^3 + [q\x] T e m4x3 - The function [-] x : M 3 M 3x3 is defined in Appendix B. All solutions of the 

l 1 1 2 ^ 

attitude kinematics satisfy the constraint ||x|| =1 driven by any w G R . The explanation is given 
as follows. It is easy to check that X X — 2 X T E (x) w = since [q] x Q = for Vg G M 3 . Therefore, 
the solutions always satisfy the constraint ||x (t)\\ 2 = 1 if ||x (0)|| = 1, t > 0. Another representation 
of attitude kinematics is 

R=[w] x R (3) 
where R G IR 3x3 is a rotation matrix satisfying the constraint R T R = I3. For ([3j, we have 

j t (R T R) = R T R + R T R = R T (\w] x + [w]ty R = 3X 3- 

That is why the evolution of R always lies on the constraint R T R = I3. 

Remark 3. The best choice of / (x) is to satisfy T u (xq) = T . However, it is difficult to achieve. For 
example, if c(x) = (xi + 1) (xi — 1), x = [xi X2\ T G M 2 , then T = { x G M 2 | x\ = l,xi = — l}. 
Since the two sets { x G M 2 | x\ = 1} and |xGM 2 |xi = — 1} are not connected, the solution of © 
starting from either set cannot access the other. Although T u (xo) 7^ J 7 , we still expect the global 
minimum x* G T u (xo) ■ That is why we often require that the initial value xo is close to the global 
minimum x*. Besides this, it is also expected that the function / (x) is chosen to make the set T u (xo) 
as large as possible so that the probability of x* G T u (xq) is higher. 

If c (x) = Ax, A G M mxn , then the function / (x) can be chosen to satisfy T ' = T u (xo) , Vxo G T . 

Theorem 1. Suppose that c(x) = Ax and / (x) = A- 1 , where A ± is with full column rank, and 
the space spanned by the columns of A is the null space of A. Then T = T u (xo) , Vxo G T . 

Proof. Since T u (xo) C J 7 , the remainder of work is to prove F C T u (xo) , Vxo G J 7 , namely 
for any x G T there exists a control input u G M. 1 that can transfer any initial state xo G T to x. 
Since xo,x G J 7 , there exist uq, u G M. 1 such that x = A^u and x (0) = A uo by the definition of 
A- 1 -. Design a control input u(t) = ~ (u — uq) , < t < i; u (t) = 0, t > i. With the control input 

above, we have x (t) — x (0) = / A^u (s) ds = A ± u (s) ds = A^u — A^uq, when t > i . Then 

_ Jo Jo _ 

x (t) = x, t > t. Hence T C (xo) , Vxo G T. Consequently, T ' = T u (xo) , Vxo G T . □ 

From the proof of Theorem I , the choice of / (x) becomes a controllability problem. However, it 

is difficult to obtain a controllability condition of a general nonlinear system. Correspondingly, it is 

difficult to choose / (x) for a general nonlinear function c (x) to satisfy T ' = T u (xo) . Motivated by 

the linear case above, we aim to design a function / (x) whose range is the null space of Vc T for any 
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fixed x G M. n . This idea can be formulated as Vi (x) = V 2 (x), where Vi (x) = {z£ M. n \Vc T z = 0}, 
V 2 (x) = {z G R n |z = /(i)u,u6 R z }. 

III. A New Projection Matrix 

The function / is the projection matrix, which orthogonally projects an vector onto the null space 
of Vc T . One well-known projection matrix is given as follows Ifll.ll2l.f7l: 

/ ( x ) = I n - Vc (x) (Vc (x) T Vc (x)) ~ Vc (x) T . (4) 

We can easily verify that Vc (x) T f (x) = 0. This projection matrix requires that Vc (x) should have 
full rank, i.e., every x G T is a regular point. However, the assumption does not hold in cases where 
Vc (x) T Vc (x) is singular. This condition is the major motivation of this paper. For a special case 
c : R n — > R, such a / (x) is designed in Theorem 2. Consequently, a method is proposed to construct 
a projection matrix for a general case c : 1" — > W 71 . Before the design, we have the following 
preliminary results. 

Lemma 1. Let Wi = {z G R n \L T z = 0} and W 2 = {z G M n |z = (l n - LL T j (s (||L|| 2 ) + ||L|| 2 

{1 x = x G R 
. Then Wi = >V 2 . 
x/0,xGR 

Proof. See Appendix C. □ 

Theorem 2. Suppose that c : R n — > R and the function / (x) is designed to be 

/(x) = /„-Vc(x)Vc(x) T /(<)-(||Vc(x)|| 2 ) + ||Vc(x)|| 2 ) . (5) 

Then Assumption 1 is satisfied with u G R n and Vi (x) = V2 (x) . 

Proof. Since c(x) = Vc(x) T x and x = f(x)u, the function / (x) is defined as in (f5]) so that 
c(x) = by Lemma 1. Therefore, Assumption 1 is satisfied with u G R ra . Further by Lemma 1, 
Vi (x) = V 2 (x) . □ 

Theorem 3. Suppose that c : R n — > R m and the function / (x) is in a recursive form as follows: 

fo = In 

fk = fk-i (in - /£-iVcfcVc£7 fc _i /(S (||/riVc fc || 2 ) + ||/riVc fe || 2 ) ) , k = 1, ■ ■ ■ ,m (6) 
Then Assumption 1 is satisfied with f = f m and u G R n and Vi (x) = V 2 (x) . 

Proof. See Appendix D. □ 

Remark 4. In practice, the unit impulse function 5 (x) is approximated by some continuous 
functions such as 5 (x) w e -7 ^, where 7 is a large positive scale. 
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IV. Controller Design and Convergence Analysis 

In this section, by using Lyapunov's method, the controller u is designed to result in v (x) < 0. 
However, the objective function v (x) is not required to be positive definite. We base our analysis 
upon the LaSalle invariance theorem lfl4l pp. 126-129]. 

A. Controller Design 

Taking the time derivative of v (x) along the solutions of (O results in 

v (x) = Vv (xf f (x) u (7) 

where Vv (x) G W 1 . In order to get v (x) < 0, a direct way of designing u is proposed as follows 

u = -Q ( x ) f (xf Vv (x) (8) 

where Q : R n ->■ R lxl and Q (x) > eli > 0, e > 0, Vx G JR n . Then © becomes 

v (x) = -Vv (xf f (x) Q (x) f (xf Vv (x) < 0. (9) 

Substituting © into the continuous-time dynamical system (O results in 

x = -f (x)Q(x) f (xfVv(x) (10) 

with solutions always satisfy the constraint c (x) = 0. The closed-loop system formed by the 
continuous-time dynamical system © and the controller ([8]) is depicted in Fig. 1 . 



Controller System 



X 

1 ► 


u = -Q(x)f(x) T Vv(x) 




x- f(x)u 

\ ) 













Fig. 1. Closed-loop control system 

B. Convergence Analysis 

Unlike a Lyapunov function, the objective function v (x) is not required to be positive definite. 
As a consequence, the conclusions for Lyapunov functions are not applicable. Instead, the invariance 
principle is applied to analyze the behavior of the solution of (fTOl . 

Theorem 4. Under Assumption 1, given xq G T, if the set K = {x G M. n \v (x) < v (xq) , c (x) = 
0} is bounded, then the solution of (fTOl) starting at xq approaches x^ G S, where S = {x G 
K\Vv(xf f (x) = 0}. If in addition Vi (xf) = V 2 (xf) , then there must exists a A* = [\\ X* 2 
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• • • A; ] t £ 1™ such that Vv (xf) = YhLi K Vc * ( x f) and c ( x f) = °' namel y x i is a Karush- 
Kuhn-Tucker (KKT) point. Furthermore, if z T V xx L (x^,X*)z > 0, for all z £ Vi (a?*) , z ^ 0, then 
is a strict local minimum, where L (x, A) = v (x) — Y^Li ( x ) ■ 

Proof. The proof is composed of three propositions: Proposition 1 is to show that K is compact 
and positively invariant with respect to (TTOb : Proposition 2 is to show that the solution of (fTOl) starting 
at xq approaches x\ 6 5; Proposition 3 is to show that i;* S 5 is a KKT point, further a strict local 
minimum. The three propositions are proven one by one in Appendix E. □ 

Corollary 1. Suppose that / (x) is chosen as © for c : 1" — > W m and the set K, = {x G 
]R n |v (a;) < u (xo) , c (x) = 0} is bounded for given xo € i 7 - Then the solution of (fTOl) starting at 
approaches x^* € 5, where 5 = {x € /C|Vi> (x) T / (x) = 0}, where x z * is a KKT point. In addition, 
if z T V xx L (x*, A*) z > 0, for all z € Vi (a;*),^ 7^ 0, then x ; * is a strict local minimum, where 
L (x, A) = v (x) - Ya=i ( x ) ■ 

Proof. Since V\ (xf) = V2 (x*) by Theorem 3, the remainder proof is the same to that of Theorem 
4. □ 

Remark 5. If K, is not a bounded set, then S defined in Theorem 4 may be empty. Therefore, the 
boundedness of the set K is necessary. For example, v (x) = x\ + 22 > s.t. c (x) = x\ — X2 = 0. The 
set /C = { x G M 2 | xi + X2 < u (xo) , xi — X2 = 0} is unbounded. According to Theorem 1, we have 
/ (x) = [1 In this case, Vv (x) T f (x) = 2^0 and then the set S is empty. 

C. A Modified Closed-Loop Dynamical System 

Although the proposed approach ensures that the solutions satisfy the constraint, this approach may 
fail if xo ^ T or if numerical algorithms are used to compute for the solutions. Moreover, if the unit 
impulse function 5 is approximated, then the constraints will also be violated. With these results, the 
following modified closed-loop dynamical system are proposed to amend this situation. 

Similar to O, we introduce the term —pVc (x) c (x) into (TTOb to result in 

x = -pVc (x) c (x) - / (x) Q (x) / (x) T Vv (x) , x (0) = x (1 1) 

where p > 0. Define v c (x) = c (x) T c (x) . Then v c (x) = — pc (x) T Vc (x) T Vc (x) c (x) < 0, where 
Vc (x) r / (x) = is utilized. If the unit impulse function 5 is approximated, then Vc (x) T / (x) w 
which can be ignored in practice. Therefore, the solutions of (fTTT) will tend to the feasible set F if 
Vc (x) is of full column rank. Once c (x) = 0, the modified dynamical system (flTT) degenerates to 
(fTOl) . The self-correcting feature enables the step size to be automatically controlled in the numerical 
integration process or to tolerate uncertainties when the differential equation is realized by using 
analog hardware. 
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V. Illustrative Examples 

A. Estimate of Attraction Domain 

For a given Lyapunov function, the crucial step in any procedure for estimating the attraction 
domain is determining the optimal estimate. Consider the system of differential equations: 

x = Ax + g (x) (12) 

where x G IR n is the state vector, A G W nxn is a Hurwitz matrix, and g : W 1 — > W 1 is a vector function. 
Let v (x) = x T Px be a given quadratic Lyapunov function for the origin of (TTZb . i.e., P G M nxn 
is a positive-definite matrix such that A T P + PA < 0. Then the largest ellipsoidal estimate of the 
attraction domain of the origin can be computed via the following equality-constrained optimization 
problem ifBl : 

min x T Px s.t. x T P [Ax + g (x)] = 0. 

xGK"\{0} 

Since {x £ W n \x T Px < x^Pxo} is bounded, the subset K, = {x G W n \ x T Px < XqPxq, 
x T P [Ax + g (x)] = } is bounded no matter what g is. 

For simplicity, consider (fT2l) with x = [x\ X2\ T G M 2 , A = —I2, P = h and g (x) = a (x) [xi 
X2] 71 , where a (x) = (xi + X2 + 2) ^(x2 + 1) — 0.1 (xi + l) 2 ^ . Then the optimization problem can 
be formulated as 

min x? + X9 s.t. fx? + Xn) a (x) = 0. 

zeR 2 \{0} v ; 

Since x ^ 0, the problem can be further formulated as 

mm.v (x) = x? + Xo s.t. a (x) = 0. 

(x 2 + 1) - 0.1 (xi + l) 2 - 0.2 (xi + x 2 + 2) (xi + 1) 



(x 2 + 1) - 0.1 (xi + l) 2 + (xi + x 2 + 2) 



Then Vv (x) = [2xi 2x 2 ] and Vc (x) = 

In this example, we adopt the modified dynamics (fTTl) . where / is chosen as (f5]) with 5 (x) = e -7 ^ , 
and the parameters are chosen as 7 = 10, p = Q = 20 /|| Vcc - ff T Wv\\ . We solve the differential 
equation (fTTl ) by using the Matlab function "ode45" with "variable-step'ij. Compared with the Matlab 

'in this section, all computation is performed by Matlab 6.5 on a portable computer (asus x8ai) with Intel core Duo 2 
Processor at 2.2GHz. 



September 25, 2012 



DRAFT 



9 

optimal constrained nonlinear multivariate function "fmincon", we derive the following comparisons: 

TABLE 1. COMPUTED RESULT FOR EXAMPLE 1 



Method 


Initial Point 


Solution 


Optimal Value 


cpu time (sec.) 


Matlab fmincon 


[-3 1] T 


[-1 -1] T 


2.0000 


Not Available 


New method 


[-3 1] T 


[0.2062 -0.8546] T 


0.7729 


0.125 


Matlab fmincon 


[2 -4] T 


[-1 -1] T 


2.0000 


Not Available 


New method 


[2 -4] T 


[0.2062 -0.8545] 7 


0.7726 


0.0940 


Matlab fmincon 


[1 -4] T 


[0.2143 -0.8533] T 


0.7740 


0.2030 


New method 


[1 -4] T 


[0.2056 -0.8550] T 


0.7733 


0.1100 



The point x s = [—1 — 1] T is a singular point, at which \7c(x s ) = [0 0] T . As shown in Table 1, 
under initial points [—3 1] T £ F and [2 — 4] T € F, the Matlab function fails to find the minimum and 
stops at the singular point, whereas the proposed method still finds the minimum. Under initial point 
[1 — 4] T ^ F, the proposed method can still find the minimum, similar to the Matlab function. Under 
a different initial value, the evolutions of (ITTT) are shown in Fig. 2. The solution is initially predisposed 
to the feasible set F. After reaching the feasible set F, the solution moves to minimize the objective 
function, all the while keeping the solution on the constraint. Once close to the singular point [— 1 
— 1] T , the solutions of (TTTT) change direction and then move to the minimum xj = [0.2061 — 0.8545] T . 
Compared with the discrete optimal methods offered by Matlab, these results show that the proposed 
method avoids convergence to a singular point. Moreover, the proposed method is comparable with 
currently available conventional discrete optimal methods and facilitates even faster convergence. The 
latter conclusion is consistent with that proposed in |H1,|151. 

B. Estimate of Essential Matrix 

For simplicity, assume that images are taken by two identical pin-hole cameras with focal length 
equal to one. Corresponding image points are represented as follows: 

mi,* = ~n— M k, m 2 , k = -tj, — M' k , k = 1, 2, • • - , N (13) 

M k,3 M k,3 

where M k ,M' k £ R 3 represent the positions of the kth point expressed in the two camera frames, 
having the relationship M k = RM' k + T, k = 1, 2, • • • , N; M k ^ : M' k 3 represent the third element 
of vectors M k} M' k , respectively. These corresponding image points satisfy the socalled epipolar 
constraint lfl~6*l p. 257]: 

ml k Em 2 ,k = 0, k = 1, 2, • • • , N (14) 
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where E = [T] x R is known as the essential matrix. By using direct product <g> and the vec 
operation, equations in ([141 ) are equivalent to 



Aip = Otvxi 



(15) 



where 



A 



rn. 



2,7V 



® m 



T 

1,N 



»7Vx9 



vec 



[T] X R). 



In practice, these image points and m2,fc are subject to noise, = 1,2, ■ 
and i? are often solved by the following optimization problem 



s.t. 



min v (x) = —ip (x) T A T Aip (x) 
l(||r|| 2 -l)=0,i(i? T fl-J 3 )=0 3 



x3 



(16) 



, N. Therefore, T 



(17) 



where x = [T T vec T {R)] T € M 12 . This is an equality-constrained optimization considered here. In 
the following, the proposed approach is applied to the optimization problem (fTTT ). By Theorem 2, 
the projection matrix for the constraint \ ( \\T\\ 2 - l) = is / = J 3 - TT T / (s (\\T\\ 2 ) + ||T| 



1 1 1 1 2 / 1 1 1 1 2 

Since ||T|| =1 has to be satisfied exactly or approximately, then 5 ( \\T 



0. So, the projection 



matrix for the constraint is / 



TT 1 



ITI 



Then the constraint can be transformed into 



f = (I3 — TT T / ||T|| 2 ) ui, where m G M 3 . By Q, the constraint UR T R-I 3 ) = 3x3 can 



be transformed into i? = [u^[ x i?, where U2 £l 3 . Furthermore, the equation above is rewritten as 
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vecf R 



(i? T ® I3) Hu2 • Then the continuous-time dynamical system, whose solutions always 
satisfy the equality constraints \ (j|T|| 2 — 1^ = and \ (R T R — J3) = 3x3 , is expressed as © 
with 

TT T J \\T\y 3x3 
9x3 {R T ® h) H 

If the initial value ||T(0)|[ 2 = 1 and R(0) T R(0) = I 3 , then all solutions of © satisfy the equality 
constraints. Since Vi> (x) = [ (R T ® J 3 ) J 3 gj [T] x ] T ^4 T ^4¥>, the time derivative of u (x) along 



/(*) 



,12x6 



U 2 



(18) 



the solutions of (O is i> (x) 

9(x) = 



ip 1 A 1 A® (x) 1 Q (x) (x) A 1 Atp < 0, where 



TT T / || T | 



H R 1 



T 



o6x9 



Choose Q(x)= fi U@ (x) yl J ^40 (x) J + e/ 6 J , where [@ (x) AQ (x) J denotes the Moore 
Penrose inverse of (x) A T AQ T (x) , and a small positive real e is to make Q (x) positive definite. 
Then (TTOb becomes 



t 



± = -fif(x) y[e (x) A 1 AS (x) 1 j +eI 6 J@(x)A i A(p(x) (19) 

where / (x) is defined in (fT8T ). 

Suppose that there exist 6 points in the field of view, whose positions are expressed in the first 
camera frame as follows: M x = [-1 1 1] T , M 2 = [2 1] T , M 3 = [1 -1 1] T , M 4 = [-1 -1 1} T , 
M 5 = [1 1 1] T , M 6 = [-1 3 1] T . Compared with the first camera frame, the second camera frame 
has translated and rotated with 

1 r 0.9900 -0.0894 0.1088 

0.0993 0.9910 -0.0894 
-0.0998 0.0993 0.9900 
The image points are generated by (fT3T ). Using the generated image points, we can obtain A 
by (O. Setting the initial value as follows T (0) = [0 1] T , R(0) = I 3 , // = 20, e = 0.01. We 
solve the differential equation (fTTb by using Matlab function "ode45" with "variable-step". Compared 
with Matlab optimal constrained nonlinear multivariate function "fmincon", we have the following 
comparisons: 

TABLE 2. COMPUTED RESULT FOR EXAMPLE 2 



f 



1 




1 


,R = 


-1 





Method 


R* T R - I 3 


cpu time (sec.) 


Matlab fmincon 


1.2469e-004 


0.2500 


New method 


1.8784e-005 


0.1400 



As shown in Table 2, the proposed method requires less time to achieve a higher accuracy. Given 
that v (x*) = 0, the solution is a global minimum. The evolvement of each element of x is shown 
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in Fig. 3. The state eventually reaches a resting state at a similar speed. With different initial values, 
several other simulations are also implemented. Based on the results, the proposed algorithm has met 
the expectations. 




Fig. 3. Evolvement of the state 



0.15 0.2 0.25 0.3 0.35 0.4 

time (sec) 



VI. Conclusions 



A continuous-time, equality-constrained optimization approach based on a new projection matrix 
is proposed for the determination of local minima. With the transformation of the equality constraint 
into a continuous-time dynamical system, the class of equality-constrained optimization is formulated 
as a control problem. The resultant approach is more general than the existing control theoretic 
approaches. Thus, the proposed approach serves as a potential bridge between the optimization and 
control theories. Compared with other standard discrete-time methods, the proposed approach avoids 
convergence to a singular point and facilitates faster convergence through numerical integration on a 
digital computer. 

Appendix 

A. Kronecker Product and Vec 

The symbol vec(X) is the column vector obtained by stacking the second column of X under the 
first, and then the third, and so on. With X = [xy] € W ixm , the Kronecker product X (g> Y is the 



matrix 



X®Y 



xuY 

XnlY 



X\mY 



XnmY 



In fact, we have the following relationships vec(XYZ) = (Z T <g> X)vec(Y) ifrTl p. 318]. 
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B. Skew -Symmetric Matrix 

The cross product of two vectors x G M 3 and y G M 3 is denoted by x x y = [x] x y, where the 
symbol [•] : R 3 -»• JR 3x3 is defined as H3j p. 194]: 








~X3 


^2 




x \x = 


^3 





-Xi 


G M 3x3 




-X 2 


Xi 








By the definition of [x] x , we have x x x = [x] x x = 03 X i, Vx G M 3 and 



vec ([x] x ) = Hx,H 



00 1 -1 

00 -1 001 

01 -1000 



d9x3 



C. Proof of Lemma 1 

Since 5 (||L|| 2 ) + ||£|| 2 = 1 if L = and 5 (||L|| 2 ) + ||L|| 2 = ||L|| 2 if L ^ 0, we have 5 
||L|| 2 7^ 0, VL G M n . According to this, we have the following relationship 



+ 



L T [l n -LL T 



+ \L\'))=L T -L T \\Lf /[& 
= 0, VL G M n . 



+ IWI 2 • 



This implies that L z = 0, \/z G W2, namely W2 C Wi. On the other hand, any 2 G Wi can be 
rewritten as 



z = /„ - LL J 



where L T z = is utilized. Hence Wi C W 2 . Consequently, Wi = W2. 

D. Proof of Theorem 3 
Denote 

Vf = {z G M n |Vcfz = 0,i = 1, ■ ■ ■ ,j,j < m\y{ = {z G R n \z = fju^uj G R n ,j < m}. 



First, by Theorem 2, it is easy to see that the conclusions are satisfied with j = 1. Assume V 1 



fc-l 



V2 1 and then prove that = Vf holds. If so, then we can conclude this proof. By Vf 



fc-l 



V2 1 (x) , we have 



Vf = {z £ R n \Vc£z = 0, z G V^ 1 } 



{z G M n |Ve£z = 0, z = fk-iuk-i, Uk-i G R n } (Vf" 1 = V 2 fc_i is used.) 



fc-i 



,fc-i 



{z G R n |Vc£/ fc _it* fc _i = 0,z = f k -iu k -i, u*_i G M n }. 
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By Lemma 1, we have 

Vc|7ft-i«fc-i = o u fc _! = (l n - /J_iVc fc Vc^/ fc _i / (<5 (||/ri Vc fc|| 2 ) + ||/ri Vc fc|| 2 ) ) u k, 

namely, Vf = {z£ R n \z = h-i (jn ~ fk-l / (<5 ( 1 1 Vc fc 1 1 2 ) + H/J^V^f) ) u fcj 

Ufc G R"} = V 2 fc . 

Proof of Propositions in Theorem 3 

(i) Proof of Proposition 1. In the space M. n , the set K, is compact iff it is bounded and closed by 
Theorem 8.2 in ifTBl p.41]. Hence, the remainder of work is to prove that K is closed. Suppose, to 
the contrary, K is not closed. Then there exists a sequence x (t n ) G JC — > p ^ K with t n — > oo. 
Whereas, t> (p) = lim u (x (t n )) < v (xo) and c (p) = lim c (x (t n )) = which imply p € JC. The 
contradiction implies that /C is closed. Hence, the set K is compact. By ©, v (x) < v (xq) with 
respect to (fTOl) . t > 0. By Assumption 1, all solutions of (fTOl) satisfy c(x) = 0. Therefore, K is 
positively invariant with respect to (ITOb . 

(ii) Proof of Proposition 2. Since /C is compact and positively invariant with respect to (fTOl) . by 
Theorem 4.4 (invariance principle) in [ 14 . p. 128], the solution of (ITOb starting at xq approaches 
v(x) = 0, namely Vv (x) T f (x) = 0. In addition, since (fTOl ) becomes x = in 5, the solution 
approaches a constant vector xf G S. 

(iii) Proof of Proposition 3. Since Vi (x*) = V2 (x*) and x ; * G 5 satisfy the following two equalities 

Vv (x?) T /(xr) = 0,e(xn = 0, 

there exists a n such that z = f{x* l )u for any z G Vi (x^) . As a consequence, for any z G 
Vi (x*) , Vv (x;*) T z = Vv (xf) T / (x*) u = 0. There must exist A* G M, i = 1, • • • , m such that 
Vt;(x / *) = EilLi \*Vci (xf). Otherwise 3z G Vi (xf), Vv{xl) T z ^ 0. Therefore, xf G 5 is a 
KKT point lfT2l p.328]. Furthermore, by Theorem 12.6 in lfT2l p.345], x* is a strict local minimum 

if z T V xx L (xf , A*) z > 0, for all z G Vi (xf) , z ^ 0. 
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